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The primary research question was: How do people's preferences vary among 
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team of educational researchers who work primarily with museums to conduct a 
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of data were collected: user exit surveys, eliciting an evaluation of the 
study site and preferred genre or type of learning activity; and server 
statistics indicating the duration of stay. Results indicated that there are 
clear differences in the type of Web-based learning activity preferred by 
adults and children. Adults are more likely to select Interactive Reference ■ 
or Simulation whereas children prefer Creative Play and Role-playing Stories. 
The adult sites yield more straightforward cognitive information, while the 
sites preferred by children allow more personal choice and interaction. 
Apparently, adults bring an intrinsic motivation to the learning experience. 
They know what they want to learn, and they want to learn it in the most 
direct way. Children, on the other hand, need to be motivated. They respond 
positively to the opportunity for interaction and choice within a goal-based 
environment that offers them an extrinsic purpose. (Contains 24 references.) 
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PAPERS 

Museums and the Web 2002 

How Do You Like To Learn? 

Comparing User Preferences And Visit Length Of 
Educational Web Sites 

David T. Schaller and Steven Allison-Bunnell, Educational Web 
Adventures, Minda Borun and Margaret B. Chambers, Museum 
Solutions, USA 



Abstract 



Developing effective public education sites for the Web requires an 
understanding of both learning theory and what appeals to learners. A 
recent study commissioned by IBM found that Web learners prefer 
passive entertainment experiences to more demanding interactive 
experiences (Karat et al, 2001). If people learn best in active modes, 
but prefer passive Web experiences, how can we develop sound 
educational activities that attract and appeal to a broad audience? 

This paper reports results of a study designed to determine people’s 
preferences for different types of Web-based educational activity. The 
primary research question was: How do people’s preferences vary 
among types of Web-based learning activity? We identified six activity 
types for comparison: Creative Play, Guided Tour, Interactive 
Reference, Puzzle/Interactive Mystery, Role-playing Story, and 
Simulation. A team of Web developers who work with museums and 
other learning sites collaborated with a team of educational 
researchers who work primarily with museums to conduct a survey of 
visitors to five different types of educational Web site. Two kinds of 
data were collected: 1 . User exit surveys, eliciting an evaluation of the 
study site and preferred genre or type of learning activity, and 2. 
Server statistics indicating the duration of stay. Results indicate that 
there are clear differences in the type of Web-based learning activity 
preferred by adults and children. Adults are more likely to select 
Interactive Reference or Simulation whereas children prefer Creative 
Play and Role-playing Stories. The adult sites yield more 
straightforward cognitive information while the sites preferred by 
children allow more personal choice and interaction. Apparently, 
adults bring an intrinsic motivation to the learning experience. They 
know what they want to learn and they want to learn it in the most 
direct way. Children, on the other hand, need to be motivated. They 
respond positively to the opportunity for interaction and choice within a 
goal-based environment that offers them an extrinsic purpose. 
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Applying Learning Theory to Interactive Media 

In the past decade, the Web has grown from a text-only tool of academia to 
a dazzling universe of ideas, community, commerce, and vanity, with a 
corresponding increase in its multimedia capabilities. How can the Web best 
be used for education? Applying learning theory to an immature medium like 
the Web is challenging, but several basic criteria for learning can safely be 
applied: 
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• “Education is not an affair of ‘telling’ and being told, but an active 
construction process”(Dewey, 1923). 

• “Learners do not learn directly from technology [or teachers, or 
books]; they learn from thinking about what they are 

doing” (Jonassen, 1999). 

• Learners must be motivated, which requires an “emotional 
connection, challenge, and payoff' (Healy, 1994). 

Somewhat more controversial is a key tenet of constructivism: “A range of 
results are possible and acceptable" (Hein, 1998). This tolerance for 
divergent outcomes distinguishes constructivism from discovery learning, in 
which “by engaging learners in activity... they will arrive at the correct 
conclusions" (Hein, 1998). Constructivism suggests that learning activities 
should allow multiple outcomes, each of which need only “'make sense' 
within the constructed reality of the learner” (Hein, 1998). (For additional 
analysis of theories of learning applied to the Web, see Schaller and Allison- 
Bunnell, 2001) 

Beyond pedagogical approach, museums and other organizations devoted to 
leisure learning must decide on the desired type of educational experience. 
Gammon (2001) offers a useful typology: 

• Cognitive: Acquire and assimilate new knowledge into existing 
schemas, apply existing knowledge, connect concepts, draw 
analogies. 

• Affective: Challenge beliefs and values, appreciate viewpoints in 
other people, inspire interest, curiosity, awe and wonder, associate 
curiosity and thinking with enjoyable experiences. 

• Social: Develop skills of co-operation and communication. 

• Developing skills (mental and physical): Prediction, deduction, 
problem-solving, investigation, observation, measuring, classification, 
testing theories, making and telling stories, decision-making, manual 
dexterity, craft skills, etc. 

• Personal: Increasing self-confidence and self-efficacy; motivating to 
investigate further. 

With these issues in mind, how has the Web fared? The Web is a form of 
interactive multimedia, or IMM. Some educational researchers and 
practitioners praise IMM’s ability to use audio, video, text, and immersive 
environments to appeal to multiple intelligences (Veenema and Gardner, 
1996). Others see in IMM the chance to move beyond passive learning 
modes and engage students in more active learning experiences (Prensky, 
2001; Crawford, 1982; Viadero, 1996; Tipping and Graesser, 1996; 

Bearman, 1997; Plowman, 1996b). 

However, actual evaluation of IMM products has shown that logistical 
problems often get in the way of fully realizing IMM's potential. Many novice 
learners have found the navigational choices offered in IMM programs 
bewildering. In studies of classroom use of IMM, students have needed 
considerable teacher assistance to make use of the programs. (Veenema 
and Gardner, 1996; Plowman, 1996b; Bearman, 1997). The problem lies in 
the very freedom afforded by IMM’s non-linear structure: “Being a user- 
controlled medium, the learner expects to have control, and yet a learner 
does not know enough to be given full control” (Laurillard, 1996). Novice 
learners need more guidance and structure to ensure that they find content 
that is both engaging and appropriate to their knowledge level. 

While classroom teachers with sufficient time, skill, and motivation can 



ERIC 



file://E:\MW2002\papers\schaller\schaller.html 



3 



5/22/2003 



overcome these difficulties and provide the necessary guidance to make use 
of the experience offered by IMM, this is not an option for Web-based leisure 
learning experiences. Web sites must attract an audience and create a self- 
contained experience that is satisfying and hopefully educational. Thus we 
must account for what people want as well as how they might learn. 

A recent study conducted by IBM suggests that, given a choice, leisure 
learners seek relief from bewildering interactive software. This formative 
research revealed that: 

“most participants did not express interest in Web sites that 
involved active interaction with the content or other people." 

They strongly preferred being “guided through an experience or 
discovery process” 



Some participants in the IBM study “viewed the more interactive design 
concepts and existing Web sites as work, not entertainment” (Karat et. al, 
2001). Indeed, the learning modes that IBM researchers offered participants 
were either quite passive (Guided Tour) or quite active (a searchable 
database of images and information, a chat room, and an online journal). 

The latter may engage the devotee or a student doing a research report, but 
can easily overwhelm those who lack an existing interest in the subject and 
the intrinsic motivation to explore it. 

Based on the results of their study, IBM developed a site featuring on-line 
tours, hosted by curators and other experts, and delivered via streaming 
video — essentially a TV-like experience with links to additional information. 
Summative evaluation of the site found that “users interacted relatively 
infrequently with the [on-line] tours, and the less they interacted, the more 
they reported feeling engaged and entertained by the experience” (Karat et. 
al, 2001). These results are disconcerting in that they contradict accepted 
learning theories that support the value of active involvement (Dewey, 1916). 

A Research Study of Web Users Preferences 

If people learn best in active modes but prefer passive Web experiences, 
how can we develop sound educational Web activities that attract and 
appeal to a broad audience? We decided to develop a detailed and focused 
pilot study of user preferences to shed more light on this complex and 
important issue. The primary research question was: How do people’s 
preferences vary among types of Web-based learning activity? 

We identified six activity types based on our previous Web development 
experience and a review of the literature (Gogg and Mott, 1993; Karat et al, 
2001; Plowman, 1996b; Sumption, 2001). The six types, as described in the 
survey instrument, were: 

• Creative Play. Draw a picture, write a story, make a movie, etc. 
Create something original based on the things you learn along the 
way. 

• Guided Tour. Join an expert to explore a topic that he or she knows 
and loves. The guide leads you on their path through the topic. 

• Interactive Reference. Explore a topic on your own, through 
informative words and pictures. Choose the links that interest you to 
find out what you want to know. 

• Puzzle/Interactive Mystery. Put on your thinking cap and solve a 



(Karat et. al, 2001) 
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puzzle or mystery. Put the clues together to discover the right answer. 

• Role-playing Story. Choose your own adventure - pick a character, 
play a role, make decisions, and see what happens. You choose your 
path through the story. 

• Simulation. Run a model of the real world and see what happens 
when you change things. The choices you make determine the 
results. 

Referring back to Gammon’s typology of learning, Interactive Reference and 
Guided Tour lend themselves primarily to cognitive learning. Creative Play, 
Puzzle/Mystery, Role-playing Story, and Simulation support both affective 
learning and developing skills. Creative Play will help learners with skills 
such as storytelling and art making. Puzzle/Mystery and Simulations with 
prediction, deduction, and other problem-solving skills. Role-playing Stories 
can challenge beliefs and values and help learners appreciate other people’s 
points of view. 

Methodology 

In November-December 2001, we conducted a series of pilot studies with 
visitors to one site in order to test various versions of the exit questionnaire. 
The challenge was to describe the types of learning activity in such a way 
that preference for type of learning activity was not confounded by 
preference for the subject matter of the particular site or its visual 
appearance. We tried and eliminated screenshots of sample sites, since 
respondents were found to cue to content and aesthetics more than the 
general activity type. Long Likert scales (5- and 7- point) seemed to confuse 
respondents, who often indicated contradictory preferences over a series of 
questions. 

Once we finalized the twelve-question survey, five activity sites previously 
developed by Educational Web Adventures, alone or in collaboration with its 
clients, were selected for this initial study to represent five of the six types of 
Web learning activity. No site exemplifying the Guided Tour was represented 
in the Eduweb portfolio. However, it remained in the list of types about which 
visitors were queried. The exit questionnaire (Appendix A) was placed on 
each of five educational Web sites. 

In addition to the exit survey, server statistics were used to determine the 
duration of stay. Summary, a log analyzer (www.summary.net), generated 
duration charts that give a clear picture of how long users spend at each site. 

A pop-up window displayed the survey on each activity site; it appeared 
when visitors came to the initial page, and remained behind the main 
browser window until the visitor clicked to leave the site. Then the survey 
returned to the foreground. The surveys were posted on the activity sites for 
10-20 days, until 50 responses from each site were collected. Table 1 
outlines the sites, types of activity, and sample size. 

A control group consisting of 299 visitors to the Educational Web Adventures 
Web site filled out the first part of the questionnaire, which dealt with learning 
in general and did not reference a particular activity. Members of the control 
group did not engage in any of the Web activities selected for the exit survey. 
The purpose of the control group was to provide a measure of user 
preferences independent of a specific learning activity for comparison to 
questionnaires filled out at the activity sites. 
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Type 


Name 


Host Srt* 


Number 


AcUvfa 


Sites 










Creative Play 


A Brush with Wildlife 


Nat. Museum of 
Wildlife Art 


SO 




interactive Reference 


Study Art 


Sanford 


SO 




PuzikfMysteiy 


Leonardo's Workshop 


Sanford 


so 




Rote*playing Story 


In Search of the Way* 
of Knowing Trail 


BrooWleld Zoo 


50 




Simulation 


Modeling Marine 
Ecosystems 
(subscription site) 


JASON Project 


so 


Control Site 










Control 


Web Adventure 
directory pa^e 


Educational Web 
Adventures 


299 



Table 1. Experimental Design 



In the following discussion, Control group results are compared to the 
Treatment population. The Treatment group consists of visitors to the five 
different Activity Sites. It is important to note that Treatment and Control are 
not used in the conventional way. We are not looking for post-treatment 
learning effects. Rather, we are comparing the preferences of users who 
have and have not experienced a particular Web activity. The purpose of the 
comparison is to be certain that observed user preferences are not solely a 
function of the activity in which they have just engaged. 



Results 



Percents 



Demographics 


Activity 

Sites 


Control 


Adults 


31 


49 


Adult Males 


II 


12 


Adult Female* 


21 


37 


Children 


69 


Si 


Boy* 


26 


22 


Girls 


42 


30 



Table 2. Demographic Characteristics of Treatment/Control 

The Activity sites had significantly more children than the Control site (X* p 
= .0001). An unexpectedly large number of adult females visited the control 
site. 





Creative 

Play 


Interactive 

Reference 


Sites (Percents) 

Puxxie/ Role- Simulation 

Mystery pbying 


Control 


Adult* 


25 


56 


32 


32 


13 


49 


Males 


6 


6 


14 


20 


7 


12 


Females 


16 


48 


16 


12 


7 


37 


Children 


75 


44 


66 


6S 


87 


5! 


Boys 


35 


10 


16 


32 


37 


22 


Girls 




34 


52 


36 


50 


30 
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Table 3 . Demographic Characteristics of Visitors by Site 



The most important differences (X 2 p = <.0001) are the large percent of adult 
females at both the Control and the Interactive Reference sites and the large 
percent of children at the Simulation site. Also the number of adult males at 
the Role-playing site was greater than expected, as was the number of girls 
at the Puzzle site. 



Percents 



Program Typo 


Activity 

Sites 


Control 


Creative Play 


24 


18 


Guided Tour 


6 


9 


Interactive 

Reference 


23 


18 


Puzzle/ Mystery 


IS 


19 


Role-playing Story 


21 


18 


Simulation 


9 


18 


Totals 


100 


100 



Table 4. Favorite Type of Computer Learning Activity 

The Activity sites differ significantly from the Control site (X 2 , p = .03). The 
respondents from Activity sites chose Creative Play and Interactive 
Reference more than expected. The Control site chose Puzzle/Mystery and 
Simulation more than expected. As will be seen below, the differences are 
probably due to the unexpectedly high percentage of adult females at the 
Control site and a significantly higher percentage of children at the Activity 
sites. 

We see immediately that, contrary to the findings of the IBM study, Guided 
Tour was the least preferred type of Web activity for both the Treatment 
(Activity) and Control sites. The reasons for the difference between this and 
the IBM study are due to differences in both sample and methodology. IBM 
used adult subjects, ages 21-55. The subjects, employees and interns at an 
IBM research facility, were recruited to evaluate selected Web sites. IBM’s 
sample sites dealt with the subject of music. In the study described here, 
children are a significant segment of the population, the user group at each 
site is voluntary and self-selected, and subject matter varies from site to site, 

With the exception of the low scores for Guided Tour, the other types of 
learning activity seem to be about equal in user preference at the Control 
site. However, significant differences emerge when the user group is 
subdivided by generation and gender. 
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Percent* 


Percent* 


Program Typo 


Adults 


Children 


Adult 

Males 


Adult 

Females 


Boys 


CM* 


Creative Play 


9 


27 


10 


9 


41 


IS 


Guided Tour 


10 


7 


10 


10 


4 


10 


Interactive Reference- 


26 


11 


20 


29 


10 


13 


Puttie/ Mystery 


21 


18 


30 


19 


12 


24 


Role-playing Story 


13 


23 


10 


14 


16 


28 


Simulation 


21 


14 


20 


20 


18 


II 


Totals 


100 


10) 


100 


100 


100 


100 



Table 5. Favorite Learning Activity by Age and Gender: Control Site 



There are a number of differences between generations (X 2 p = .0001). 
Adults preferred Interactive Reference and Simulation and children preferred 
Creative Play and Role-playing. 

Gender differences between adults were not significant. However, 
differences between boys and girls were significant (X 2 p = .01 ). Boys 
showed a preference for Creative Play whereas girls favored Role-playing 
Story and Puzzle/Mystery. 





Percents 


Percents 


Program Type 


Adults 


Children 


Adult 

Males 


Adult 

Females 


Boys 


Girls 


Creative Play 


9 


31 


10 


8 


33 


31 


Guided Tour 


15 


5 


15 


15 


4 


6 


Interactive Reference 


40 


14 


25 


46 


II 


16 


Puzile/Mysiery 


13 


17 


20 


10 


14 


19 


Role-playing Story 


16 


23 


25 


13 


29 


19 


Simulation 


7 


10 


5 


8 


10 


10 


Totals 


100 


100 


too 


100 


100 


100 



Table 6 . Favorite Learning Activity by Age and Gender: Activity Sites 



Differences in program type preferences at the Activity sites are similar to 
those found at the Control site. Again there are significant differences 
between generations (X 2 p = <.0001). As at the Control site, adults prefer 
Interactive Reference and children prefer Creative Play and Role-playing. At 
the Activity sites, gender differences were not statistically significant. 
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Sites (Percent*) 


Program 

Type 


Creative 

Play 


Interactive 

Reference 


Puzzle/ 

Mystery 


Role- 

playing 


Simu- 

lation 


Control 


Creative Ptay 


45 


10 


28 


16 


23 


18 


Guided Tour 


10 


6 


3 


14 


6 


9 


Interactive 

Reference 


13 


65 


(5 


A 


12 


18 


Puzzfe/Mystery 


10 


8 


21 


25 


12 


19 


Role-playing 

Story 


10 


6 


18 


35 


32 


18 


Simulation 


5 


A 


15 


6 


15 


18 


Totals 


100 


100 


100 


100 


IOO 


100 



Table 7. Favorite Learning Activity by Respondent's Site 



There are significant differences in program type preferences among the 
Activity sites (X 2 p = <.0001). As mentioned above, with the exception of a 
lack of interest in Guided Tour, respondents at the Control site were almost 
evenly divided in their preference for program type. The Activity site 
respondents, on the other hand, tended to prefer the type of program they 
were using. This is notably the case with Creative Play, Interactive 
Reference and Role-playing Story. 

It is important to remember that the Control site data was collected as a 
check on the tendency of Activity site users to prefer the type of program 
they are using. However, the similarity in preferences between Control and 
Activity sites (when subdivided by generation and gender) suggests that 
there are patterns in preferences that transcend the particular site. 



Percents 



Location* 


Activity 

Site* 


Control 


Frfcnd't House 


2 


1 


Home 


40 


40 


Library 


1 


2 


School 


48 


42 


Work 


7 


It 


Other* 


2 


A 


Total* 


100 


100 



Table 8. Location of Respondents 



*Home school (3), Internet cafe (2) Work at Home (1), Grandparent’s house 
(1 ), No answer (9). Numbers in parentheses () indicate number of 
responses. 

There were no significant differences between the Activity sites and the 
Control site in terms of where the respondents were when using the 
computer. Home and School were the most frequent location for both Activity 
and Control. 

The remainder of the questionnaire dealt with users’ responses to the Web 
activity in which they had engaged; consequently, these questions were not 
asked at the Control site. The tables below show results from the five Activity 
sites. 
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Creative 

Play 


Interactive 

Reference 


Percent! 

Puzzle/ 

Mystery 


Role- 

Paring 


Simulation 


Friend's House 


2 


2 


2 


2 


29 


Home 


13 


76 


S9 


40 


10 


Library 


0 


0 


2 


0 


2 


School 


73 


12 


35 


44 


79 


Work 


13 


6 


2 


12 


2 


Other 41 


0 


4 


0 


2 


4 


Totali 


ICO 


100 


100 


100 


100 



Table 9. Location of Respondents by Site 



The differences are significant (X 2 p = <.0001). Interactive Reference and 
Puzzle/Mystery are accessed most from home while Creative Play and 
Simulation are often school activities. 





Creative 

Play 

(Art) 


Interactive 

Reference 

(Art) 


Percents 

Puzzle/ 

Mystery 

(History) 


Role- 

playing 

(Natural 

History) 


Simulation 

(Ecology) 


Little 


40 


18 


48 


41 


28 


Medium 


aa 


43 


24 


45 


50 


lots 


23 


39 


28 


14 


22 


Totals 


100 


100 


100 


100 


100 



Table 10. Previous Knowledge of Topic 



More people than expected indicated they knew “lots” for the Interactive 
Reference site and more indicated “little" for the Puzzle/Mystery site ( X 2 p 
= . 01 ). 





Creative 

Play 


Interactive 

Reference 


Percents 

Puzzle/ 

Mystery 


Role- 

playing 


Simulation 


Assigned by teacher 


69 


16 


26 


28 


77 


Personal interest 


6 


20 


28 


36 


5 


Professional interest 


10 


8 


6 


6 


2 


Recommended by friend 


2 


0 


2 


2 


0 


Recommended by 
parent 


0 


0 


4 


4 


2 


To use in a lesson 


10 


36 


17 


9 


5 


Other 1 ' 


2 


20 


17 


13 


9 


Total* 


100 


100 


100 


100 


100 



Table 1 1. Why Doing This Web Activity? 
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Creative Play: to show son (1). 

Interactive Reference: for a project (4), info about artist (1), daughter’s 
homework (1), find a picture (1). 

Puzzle/Mystery: fun (1), no answer (6). 

Role-playing Story: fun (3), exploring the Web (2) recommended by 
teacher (1). 
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Simulation: no answer (5). 



Teacher assignments are responsible for a high percentage of Creative Play 
and Simulation usage (X 2 p = < .0001). This corresponds with Table 9 that 
shows that these activity sites are most often used from school. Personal 
interest was unexpectedly high for Role-playing and the Interactive 
Reference site was most often accessed to use in a lesson. 



Percents 


Decreased 


12 


Remained the same 


44 


Increased 


43 




Total ICO 



Table 12. Has your Enthusiasm for this Topic Changed? 

Most people reported that their interest in the subject matter had stayed the 
same or increased after the Web-based learning activity. Results were not 
significantly different from site to site. 





Creative 

Play 


Interactive 

Reference 


Percents 

Puzzle/ 

Mystery 


Role* 

playing 


Simulation 


Finished k 


40 


54 


48 


75 


45 


Got bored 


9 


4 


21 


9 


23 


Got confused 


2 


0 


S 


2 


10 


Ban out oi Time 


24 


20 


18 


9 


10 


Other* 


4 


22 


9 


6 


13 


Totals 


100 


100 


100 


100 


100 



Table 13. Why Respondents Left the Site 



* Creative Play: don’t have Java (1), no answer (1). 

* Interactive Reference: got what was wanted(4), didn’t get what was 
wanted (3), Knew everything (1), going to play art game (1), no answer (3). 

* Puzzle/Mystery: couldn’t find answer (1 ), done with it (1 ), just got started 
(1), text too big (1). 

* Role-playing Story: slow (1), not what I was looking for (1) finishing school 
work (1). 

* Simulation: no answer (6). 

The differences here are significant (X 2 p = .007). More respondents than 
expected left the Simulation site because they were confused. Respondents 
tended to finish the Role-playing Story more often than expected and 
respondents left the Interactive Reference site for “other” reasons. The 
reasons were likely to be that the information sought was found or not found. 
Actually “found what was wanted” might also be interpreted as “finished”, 
which if coded that way would increase the percent in Interactive Reference 
who felt they had “finished it”. 
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The two sites with the highest percentage of users who “finished” the 
experience are Creative Play and Role-playing story. These are also the 
sites that are most preferred by children. Perhaps the goal-based structure 
with a clear ending is part of the appeal of the two sites. 



Percent* 


Worse 


IS 


Equal 


39 


Better 


45 




Total 100 



Table 14. How does this Activity Compare to Others of Its Type? 

Most people felt that the Activity site was as good or better than other similar 
sites. There were no significant differences from site to site in users’ 
comparison of the quality of the site. 

Additional data was gathered in the form of server logs, which were analyzed 
for duration information 



Program Type 


Idoal 

Time 


Moan Time 


Mean Time 
2* (minus 
outliers) 


Percent of 
Ideal 

(Mean 2/1 deal) 


Creative Play 


21 min 


6 min 19 tec 


7 mil 48 sec 


37 


Interactive Reference 




7 min 25 tec 


7 win 54 tec 


- 


Punk/Mystcry 


20 min 


5 min 29 sec 


7 min 42 sec 


38 


Role-playing Story 


19 min 


7 min 46 sec 


1 2 min 24 sec 


65 


Simulation 


2$ min 


1 6 min 40 sec 


1 4 min 42 tec 


59** 



^Outliers (under 8 seconds or an hour or more) are eliminated. 

'** This activity required written journal entries. 

Table 15. Time at Site 

We made a crude measure of the “Ideal Time” it takes a person to go 
through the activity by timing an adult clicking on and reading all the 
available content. We electronically collected actual time spent at the site. 
The mean is given first for the whole group and then minus the outliers, 
excluding those who left immediately (presumably because the site did not 
offer what they were looking for or did not have the required technology) and 
those who stayed more than an hour (possibly because they left their 
browser on the site after finishing the activity). 

A comparison of mean time to Ideal Time shows that unusually long times 
were spent at the Role-playing Story and Simulation. However, many 
children were engaged in the Simulation Activity because their teacher 
assigned it to them (see Table 11); they were required to complete the 
Simulation and submit journal entries with their conclusions. 

Apart from the Role-playing Story and Simulation, the mean times are very 
similar, indicating that these activity types have comparable holding power. 
The duration charts that follow reveal in more detail the differences in holding 
power of the various activity types. 
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Duration of Visit: Creative Play 
(A Brush with Wildlife) 

Average Visit: 7 minute* 48 second* 



Timo Range 


Count 


0 


11 — m 


1 MC 


24 mmmmmm 


2-3 tsci 


23 mmmmmmm 


4-7 mci 


21 mmmmmm 


8-14 ttci 




15-29 mci 




30-59 leci 




1 mm 




2-3 rani 




4-7 ami 




3- 14 ram 




13- 29 ram 




30-59 nnu 


19 


1 heur 


1 ■ 



Duration of Visit: Puzzle/Mvfttery 
(Leonardo's Workshop) 

Average VU«:7 minute* 42 second* 



Time Range 


Count 


0 


132 


1 ICC 


81 


2-3 tec* 


138 


4-7 sect 


96 


8*14 tec* 


77 


15-29 »cc» 


84 


30-59 ices 


102 


1 ram 


110 


2-3 trail 


136 


4-7 mm* 


125 


8-14 rani 


114 


13-29 ram 


130 


30-59 tram 


34 


1 hour 


8 


2-3tu>itri 


3 




Duration of Visit: Interactive Reference/ 
Encyclopedia (Study Art) 

Average Visit; 7 minute* 54 iccond* 



Time Range 
0 

1 ttc 
2*5 isct 
4-7 ices 
8-14 tec i 
15-29 teci 
30-39 teci 
1 ran. 

2*3 mini 
4-7 mini 
8*14 ran* 
15-29 mat 
30-59 ram 
l how 
2-3 heart 




Duration of Visit: Role-Playing Story 
(In Search of the Ways of of Knowing Trail) 

Average Visit: 12 minute* 24 second* 



Time Range 
0 

4-7 net 
8-14 tecs 
15-29 »*« 
30-59 sec* 

I min 
2-3 owa 
4-7 mbs 
8- 14 min* 
15-29 mm* 
30-59 mini 



Count 

7 

105 

45 

11 

5 

13 
9 

14 



66 ^ 



Duration of Visit: Simulation 
(Modeling Marine Ecosystems) 



Average Visit: 14 minute* 42 second* 



Time Range 


Count 


0 


11 


m 


1 tec 


12 


■ 


2-3 ecci 


17 


m 


4-7 tec* 


22 


an 


8-14 tec* 


33 


Ml 


15-29 mc * 


30 




30-39 tec* 


56 




1 ran 


62 


mmmm 


2-3 mm* 


104 


ma—mmmm 


4-7 ram 


134 




8-14 ram 


144 




15-29 trail 


246 




30-39 ran* 


127 




1 hour 


30 





Table 16: Duration of Visit 
(detailed image) 



At most sites, with the exception of Creative Play and Simulation, a sizeable 
number of visitors leave within the first seven seconds. The explanation for 
this finding may lie in the specifics of the individual sites rather than in their 
Activity type. 

The Marine Ecosystems Simulation was assigned as schoolwork for three- 
quarters of the users of the site. The Creative Play activity starts with a 
series of animations that may hook visitors more effectively than the text 
introductions to the Interactive Reference and Puzzle/Mystery sites. The 
Role-playing site begins with a splash/Flash plug-in detection page that turns 
away a sizeable percentage of visitors apparently due to the wait involved, 
rather than lack of the Flash plug-in (only 16% of visitors did not have the 
plug-in). 

The remainder of each chart is more revealing. Interactive Reference shows 
a bell curve. The other sites have skewed distributions, indicating greater 
holding power, after the initial drop-off. Most striking is the curve for Role- 
playing Story. Half of the visitors who got past the splash page (and 25% of 
all visitors) stayed for at least fifteen minutes, approaching and even 
surpassing the ideal time (19 minutes) for that site. The curves for Creative 
Play and Puzzle/Mystery are less dramatic, but suggest a similar pattern. 



Conclusions 
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Comparisons of users’ responses to an exit questionnaire posted on five 
Web-based learning activity sites and at a control site answer some 
fundamental questions about users preferences for different types of Web- 
based learning. An earlier study of this subject conducted by IBM found that 
adult users prefer Guided Tours or non-interactive Web experiences. In the 
current study, using self-selected subjects and including a large proportion of 
children, Guided Tour was the least preferred type of Web activity for both 
the Treatment (Activity) and Control sites. 

With the exception of the low scores for Guided Tour, the other types of 
learning activity seem, at first, to be about equal in user preference at the 
Control site. However, when the user group is subdivided by generation and 
gender, significant differences emerge having to do with the user’s age, 
location and purpose for engaging in the activity. 

There are significant age differences in preferences. At both the Activity and 
Control sites, adults prefer Interactive Reference while children prefer 
Creative Play and Role-playing Stories. Adult females are over-represented 
at the Control site. Judging by the Activity sites, many of these adult females 
may be teachers who use Web sites for their lessons. The Creative Play and 
Simulation sites used here are often school activities and are usually 
assigned to the children by a teacher. 

Activity site respondents tended to prefer the type of program they were 
using. The Control site data was collected to counter this tendency. The 
similarity in preferences between Control and Activity sites (apart from 
differences in demographic composition) suggests that the observed 
preference patterns transcend particular sites. 

Most people reported that their interest in the subject matter had stayed the 
same or increased after the Web-based learning activity. Also, most users 
felt that the Activity site was as good as or better than other similar sites. 

This suggests that production values did not heavily influence preference for 
activity type. 

Implications of the Web Learning Preference Study 

There are clear differences in the type of Web-based learning activity that 
adults prefer in comparison to children. Adults prefer the information-based 
activities of Interactive Reference and Simulation, whereas children, not 
surprisingly, are more inclined to prefer the exploratory experiences of Role- 
playing Story and Creative Play. The adult sites yield more straight-forward 
cognitive information while the sites preferred by children have strong 
affective components and allow more personal choice and interaction, but 
can lead to “dead ends” or less utilitarian solutions. Apparently, adults bring 
an intrinsic motivation to the learning experience. They know what they want 
to learn and they want to learn it in the most direct way. Children, on the 
other hand, need to be motivated. They respond positively to the opportunity 
for interaction and choice within a goal-based environment that offers them 
an extrinsic purpose. 

Goal-based environments are advocated by Roger Shank, director of the 
Institute for the Learning Sciences at Northwestern University. He describes 
them as “Goal-Based Scenarios” (GBS) — structured learning programs that 
can be successful in both physical and virtual environments. The goals in 
these scenarios are not arbitrary extrinsic motivations, such as a good test 
score, prize, or reward. Rather, they stem from the activity itself — solve a 
crime, reach a destination, create an original artwork — which reinforces the 
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cognitive goals of the activity. Thus, GBSs “provide motivation, a sense of 
accomplishment, a support system, and a focus on skills rather than 
facts” (Schank, 1992). In this way, they meet the basic criteria for learning of 
Dewey, Jonassen, and Healy cited above. They create an environment for 
doing and thinking, and provide both a challenge and a payoff. If designed 
properly, they can also connect with pre-existing knowledge and help forge 
an emotional connection with the subject matter. 

Of the six types of Web-based learning activities explored here, two 
(Puzzle/Mystery and Role-playing Story) are naturally suited to the GBS 
approach; they inherently provide a motivation to reach a solution. Creative 
Play and Simulation, on the other hand, may or may not establish a clear 
goal. If designed as a GBS, each of these four activity types offers a goal or 
challenge, a payoff, structure and guidance, and some degree of 
interactivity. Young or novice learners who are unfamiliar with a particular 
learning domain need such guidance and structure to attract and hold their 
attention. It is interesting to note that in the Creative Play, Role-playing Story 
and Puzzle/Mystery activities, a plurality of users indicated they knew “little” 
about the subject to begin with (Table 10). 

In contrast, Guided Tour and Interactive Reference are not goal-based 
scenarios. There is no payoff or achievement for completing the activity; 
learners must bring their own intrinsic motivation to the task. Users of these 
Web sites may have more expertise in the subject as well. On the Interactive 
Reference site, more people than expected indicated they knew “lots” about 
the subject (Table 10). 

Within the structure and guidance provided by GBSs, young learners prefer 
some degree of freedom. Creative Play and Role-playing Stories, both 
preferred by children, offer a series of choices in the path of the activity and 
some control over the outcome. Puzzle/Mystery, which was less favored, 
and Guided Tour, the least popular type, offer only one outcome and less 
opportunity for personal involvement. 

This brings us to another important dimension: whether the outcome is 
determined (created by the site developers) or user-created. Different 
pedagogies underlie the two types of outcome. Discovery learning lends 
itself to puzzles and mysteries, with their single correct solution (determined), 
while constructivism supports user-created outcomes that allow more 
personal choice and involvement. Most of the activities rely on determined 
outcomes, but Creative Play, and to a lesser extent Simulation, permit 
learners to create their own outcomes, be it a picture, a story, or a unique 
configuration of the variables in the simulation. 

Putting the Results into Practice 

First, developers of educational Web activities must decide whether their 
primary audience is adults or children, since the two groups have different 
learning preferences. Adults prefer reference sites. Children prefer goal- 
based scenarios, particularly Creative Play and Role-playing Stories. These 
activity types offer an appealing middle ground between the highly structured 
and constrained Guided Tour approach, and the “explore according to your 
own interests" reference format typical of much interactive multimedia. If a 
site must appeal to both children and adults, developers should consider a 
dual approach, combining reference and play (as in Leonardo’s Workshop 
and Study Art). 

Second, developers must decide on a pedagogical approach. Guided Tour 
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and Interactive Reference sites typically provide a traditional 
expository/didactic approach to learning. Puzzle/Mystery activities reflect a 
discovery learning orientation, in which a single correct solution or 
conclusion is the goal. Creative Play takes a constructivist approach, 
encouraging open-ended experience. Simulations and Role-playing Stories 
can take a variety of forms, from discovery to constructivism. 

Developers should also consider their audience’s expertise in the subject. 
Expert learners with existing interest in the domain are more likely to favor 
interactive reference sites. Novice learners, regardless of age, are more 
likely to need and prefer a guided experience to introduce them to the 
subject and motivate them to learn more about it. 

The audience’s degree of expertise also affects the learning goals of a Web 
activity. For novices, affective learning experiences can inspire interest and 
curiosity in a subject, while skill-building activities help them to develop the 
ability to pursue further understanding. For experts who already have the 
knowledge base and skills to tackle the subject, interactive reference sites 
can provide a satisfying cognitive learning experience. For novices, goal- 
based scenarios can combine affective and skill building learning 
experiences with cognitive learning. 

Of course, just because a Web activity attracts and holds users’ interest 
doesn’t mean it is achieving its educational goals. Evaluating learning 
outcomes is no easy task, and a matter for another study. However, a Web 
activity or any other learning activity must first attract and hold the interest of 
learners in order to have the opportunity to achieve its learning objectives. 

Appendix A: Web Activity Preference Survey 

When you are done exploring the Study Art section of this Web site, we 
would like to ask you a few questions. Your responses will help us develop 
better Web sites in the future. Your responses are anonymous and will be 
kept confidential. 

Please complete this survey when you are leaving Study Art. 

Are you: 

Male 

Female 

How old are you? 



5-8 

9-10 

11-13 

14-18 

19-22 

23-35 
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36-49 



50 + 

Where are you now? 

Home 

School 

Work 

Library 

Friend’s house 
or Other: [enter text] 

How much did you know about art before you came to this Web site? 

Very little 
Medium 
A lot 

Why were you looking at this Web site? 

Assigned by teacher 
Recommended by a friend 
Recommended by parent 
To use in a lesson 
Personal interest 
Professional interest 
or Other: [enter text] 

Why are you leaving this Web site? 

Found what I was looking for 
Ran out of time 
Got bored 
Got confused 
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or Other: [enter text] 



Study Art is an Interactive Reference/Encyclopedia Web site. How does 
it compare to other such sites you’ve seen on the Web? 

Much worse 

Equal to 

Much better 

What would you say Study Art is about? 

[enter text] 

What is one thing you learned that you didn’t know before? 

[enter text] 

Now that you’re leaving Study Art, has your enthusiasm for art 
changed? 

Decreased 

Stayed the same 

Increased 

What is your favorite type of computer learning activity? There are six 
types of activities below. Please choose one: 

Guided tour. Join an expert to explore a topic that he or she knows and 
loves. The guide leads you on the path they chose through the topic. 

Interactive reference/encyclopedia. Explore a topic on your own, through 
informative words and pictures. Choose the links that interest you to find out 
what you want to know. 

Role-playing story. Choose your own adventure — pick a character, play a 
role, make decisions, and see what happens. You choose your path through 
the story. 

Creative play. Draw a picture, write a story, make a movie, etc. Create 
something original based on the things you learn along the way. 

Simulations. Run a model of the real world and see what happens when 
you change things. The choices you make determine the results. 

Puzzle or interactive mystery. Put on your thinking cap and solve a puzzle 
or mystery. Put the clues together to discover the right answer. 



If you have any comments about this survey or Study Art, please write 
them here: 
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[enter text] 
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